Repository navigation
Conversation
An official stage-1 race (contracts-net-loss stall at block 831, run 39866725719, analysed by Fable): each node sends its hash when it takes a bootstrap table; a hash that reaches a node before that node took the table is dropped (getHash returns when not a confidant, or recv_hash is cleared by onRoundStart(..., false)), so the last node to take the table never gets MinTrustedNodes-1 hashes, and with round_elapse_time below the ~12.5 s fallback chain the bootstrap repeats for ever. - getHash: a node that is not a confidant verifies and hands to the solver a hash of the round after its last block (the solver caches it, NormalState ignores it); other rounds dropped as before - getBootstrapTable: a table of the round the node is in keeps the received hashes (onRoundStart(..., true), as the starter already does), replayed by gotConveyerSync - SolverCore::gotHash: one entry per sender, so kept hashes do not pile up across retries - bootstrap-livelock becomes a regular robustness scenario Every check of a hash runs as for a confidant; the block is built from the same stage messages, so a mixed network does not fork. Consensus-relevant: to be merged only with the maintainer's OK. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…race # Conflicts: # .github/workflows/robustness.yml # docs/CHANGES.md
…00 blocks); CHANGES.md Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Closed without merging. The debug logs of the captured livelock (run 39902923745, scenario bootstrap-livelock-debug) show that the stall starts from a fork of block 1433: two valid blocks that differ only in realTrustedMask, created by a stage-3 retry race. No hash was lost; the stuck nodes rejected matching-sequence hashes because they hold the other block. Keeping hashes across a same-round bootstrap, which is what this PR does, does not address that. The branch is kept. The fix will be a separate PR, after the maintainer's decision. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Consensus-relevant: do not merge without the maintainer's explicit OK. Status: partial hardening, does not fix the livelock.
What it does
getHash: a node that is not a confidant verifies and caches a hash of the round after its last block. The official node drops it.getBootstrapTable: a table of the current round keeps the hashes already received.SolverCore::gotHash: keeps one entry per sender.Every check on a hash runs exactly as for a confidant, and the blocks are built from the same stage messages, so a mixed network does not fork. This was confirmed with
node_alt_ref=masteroncontracts-net-loss,net-partition,mixed-power-cut,net-loss,chaos-mixedandbootstrap-livelock: equal state digests.What it does not do
With #77 on every node,
bootstrap-livelockstill stalled once (run 39886642406, R-964.1). Fable's analysis of that run:The hash race this PR fixes was not exercised: no node received a hash before its table.
The real livelock is a timing problem in official behaviour:
round_elapse_time;Master without csnode, solver: keep stage-1 hashes across a same-round bootstrap (partial hardening; consensus-relevant) #77 shows the same pattern (run 39866725719, R-832.1).
Next
bootStrapWeight_on "Bootstrap off".round_elapse_time, with debug logs on chosen nodes (harness PR).This PR can be kept as hardening or closed. It is not needed for the livelock fix.
🤖 Generated with Claude Code